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ABSTRACT 


Within  the  Operational  Simulation  and  Analysis  (OS&A)  branch  of  the  U.S.  Army 
Armament  Research,  Development,  and  Engineering  Center  (ARDEC)  at  Picatinny 
Arsenal,  there  exists  no  standard  model  for  development  and  execution  of  an  Analysis 
Project  Plan.  A  project  plan  is  a  formal  document  which,  when  agreed  upon  by  parties 
involved,  guides  the  execution  and  control  of  a  project. 

Having  such  a  plan  is  important  to  the  OS&A  branch  and  ARDEC  as  a  whole 
because  it  documents  decisions,  facilitates  communication  among  stakeholders,  and 
maintains  a  record  of  scope,  cost,  and  schedule  baselines.  By  instituting  a  standardized 
process,  the  OS&A  branch  would  ensure  that  results  based  on  the  Analysis  Project  Plan 
are  reusable,  allow  for  configuration  management,  better  management  of  overall 
resources,  and  better  validation  and  verification. 

Through  Systems  Engineering  principles,  personal  observations,  team 
collaboration,  and  other  considerations,  the  process  proposed  in  this  thesis  has  been 
developed  for  the  Analysis  Project  Lead  to  improve  his  or  her  ability  to  systematically 
accomplish  the  job.  Ultimately,  the  proposed  process’s  intent  is  to  establish  a  flexible 
process  where  communication  of  the  problem  is  precise,  the  magnitude  of  the  solution  is 
relevant  and  reliable,  and  the  tools  and  personnel  to  execute  the  analysis  are  employed  at 
the  right  times. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Within  the  Operational  Simulation  and  Analysis  (OS&A)  branch  of  the  U.S. 
Anny  Armament  Research,  Development,  and  Engineering  Center  (ARDEC)  at  Picatinny 
Arsenal,  there  exists  no  standard  process  for  development  and  execution  of  an  Analysis 
Project  Plan.  A  project  plan  is  a  formal  document  which,  when  agreed  upon  by  parties 
involved,  guides  the  execution  and  control  of  a  project.  The  Analysis  Project  Plan  should 
leverage  the  Defense  Acquisition  University’s  definition  of  a  Systems  Engineering  Plan 
(SEP),  which  is  “A  detailed  formulation  of  actions  that  should  guide  all  technical  aspects 
of  an  acquisition  program”  (Defense  Acquisition  University  2011).  The  SEP  is  intended 
to  be  a  roadmap  that  supports  program  management  by  defining  comprehensive  systems 
engineering  activities  and  is  prepared  for  each  phase  of  a  Defense  Acquisition 
Framework  (Defense  Acquisition  University  2011). 

Specifically,  an  Analysis  Project  Plan  uses  the  same  processes  as  the  SEP  and  is 
further  tailored  to  the  expertise  of  the  OS&A  branch.  Having  such  a  plan  is  important  to 
the  OS&A  branch  and  ARDEC  as  a  whole  because  it  documents  decisions,  facilitates 
communication  among  stakeholders,  and  maintains  a  record  of  scope,  cost,  and  schedule 
baselines.  By  instituting  a  standardized  process,  the  OS&A  branch  would  ensure  that 
results  based  on  the  Analysis  Project  Plan  are  reusable,  allow  for  configuration 
management,  better  management  of  overall  resources,  and  better  validation  and 
verification. 

Through  systems  engineering  principles,  personal  observations,  team 
collaboration,  and  other  considerations,  the  process  proposed  in  this  thesis  has  been 
developed  for  the  Analysis  Project  Lead  to  improve  his  or  her  ability  to  systematically 
accomplish  the  job.  Ultimately,  the  proposed  process’s  intent  is  to  establish  a  flexible 
process  where  communication  of  the  problem  is  precise,  the  magnitude  of  the  solution  is 
relevant  and  reliable,  and  the  tools  and  personnel  to  execute  the  analysis  are  employed  at 
the  right  times. 
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B. 


ARDEC  AND  OS&A  ORGANIZATIONAL  RELATIONSHIP 


ARDEC’s  mission  is  to  provide  world  class  support  for  the  research, 
development,  production,  field  support  and  demilitarization  ARDEC  and  RDECOM 
products.  Concurrently,  ARDEC  also  strives  to  support  the  RDECOM  mission  of  getting 
the  right  technology  to  the  right  place,  at  the  right  time  for  the  War  fighter  (U.S.  Anny 
ARDEC  2001). 


Systems 

Engineering 

Competency 


Operational 
Simulation  & 
Analysis  Branch 


Figure  1.  Research,  Development  &  Engineering  Command  (RDECOM)  to  the 
Operational  Simulation  &  Analysis  (OS&A)  branch  Hierarchy 


As  seen  in  Figure  1,  the  Operational  Simulation  and  Analysis  (OS&A)  branch  is  a 
subset  of  the  Armaments  Research  Development  and  Engineering  Center  (ARDEC) 
under  the  Systems  Engineering  Competency  and  System  Analysis  Division.  The  vision  of 
the  OS&A  branch  is  to  provide  a  processing  &  simulation  facility  which  integrates 
engineering,  operational,  logistics,  and  visualization  capabilities  and  products  through  the 
use  of  distributed  simulation  technologies  with  the  ultimate  goal  of  conducting 
operational  analyses  of  ARDEC  technology  concepts.  These  analyses  benefit  the  war 
fighter  by  validating  their  requirements  early  in  the  life  cycle  and  quantifying  their 
benefits.  The  ultimate  intent  of  the  OS&A  branch,  in  support  of  the  ARDEC  and 
RDECOM  mission  is  to  aid  in  providing  shorter  acquisition  cycle  times  of  ARDEC 
armament  products  (F.  J.  Luzzi,  personal  communication,  January  2010). 

C.  PURPOSE  OF  A  STANDARDIZED  PROCESS 

Various  projects  within  the  OS&A  branch  have  used  and  provided  insight  to  the 
need  for  a  standardized  process.  Currently,  a  prototype  process  is  being  applied  to  a 
Homeland  Defense  (HLD)  Project  which  requires  the  OS&A  branch  to  provide  a 
simulation  environment  for  the  evaluation  of  ARDEC  products  (direct  requirement  from 
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the  customer)  with  a  focus  on  Disaster  Planning  &  Response  (derived  requirements  from 
stakeholders  and  subject  matter  experts  identified  by  the  primary  customer). 

Collectively,  the  OS&A  branch  would  like  to  employ  a  Systems  Engineering 
process  tailored  for  their  specific  capabilities.  Justification  for  such  a  process,  as 
discussed  in  detail  in  Chapter  II,  includes  the  need  for  Reuse,  Configuration  Management 
(CM),  Resource  Management,  Verification  &  Validation  (V&V),  as  well  as  establishing 
methods  to  develop  accurate  time  and  cost  estimates. 

Since  previous  project  efforts  tend  to  be  applicable  to  existing  and  future  projects, 
establishing  a  configuration  management  scheme  to  archive  past  analyses  provides  the 
branch  the  ability  to  positively  affect  cost,  schedule,  and  perfonnance  criterion. 
Additionally,  such  configuration  management  processes  contribute  to  the  V&V  of 
projects.  Through  use  of  a  configuration  management  scheme,  Developers,  Analysts, 
Project  Leads,  and  others  will  be  able  to  review  items  stored  and  obtain  critical 
information  which  supports  the  managed  product’s  integrity,  authenticity,  non¬ 
repudiation,  verification,  and  validation.  Perhaps  most  important  is  employing  the 
process  to  capture  subject  matter  expert  feedback  on  the  project  while  it  is  in  progress. 
Over  time,  this  process  will  enable  the  team  to  build  up  a  repository  of  simulation  and 
analysis  projects.  By  doing  so,  future  projects  are  likely  to  require  shorter  lead  times 
given  applicable  historical  analyses  conducted. 

Finally,  use  of  the  process  will  provide  clear  definitions  of  responsibility  since  not 
every  member  in  a  project  has  actions  at  the  same  time  and  resource  scheduling  within 
the  process  should  reflect  this  fact.  By  resourcing  the  project  within  process  guidelines, 
management  will  be  able  to  better  manage  their  resources  and  will  work  towards 
maximizing  productivity. 

The  current  proposed  process  consists  of  numerous  steps  grouped  into  four 
functional  categories:  Initiation,  Planning,  Execution,  and  Analysis.  It  is  of  particular 
importance  to  note  that  the  phases  stated  above  were  not  in  the  original  proposed  process. 
This  is  because  it  was  not  until  the  original  process  was  put  into  practice  on  various 
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projects  that  clear  phase  delineation  became  apparent  thus  illustrating  the  intended 
fluidity  of  the  process.  Figures  2  and  3  depict  the  evolution  of  the  proposed  process  over 
the  course  of  researching  this  thesis. 


Initiation  Phase 


Planning  Phase 


Execution  Phase 


Figure  2.  Initial  Proposed  Architectural  Process 
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Figure  3.  Current  Proposed  Architectural  Process 


While  the  process  exists  using  pre-established  phases,  steps,  and  personnel  roles, 
it  should  in  no  way  be  taken  to  mean  that  no  other  phases,  steps,  or  personnel  roles  can  be 
added,  deleted,  or  modified.  Having  the  ability  to  adjust  the  process  as  necessary  is  a 
critical  condition  to  the  success  of  employing  the  process  itself. 

Finally,  it  is  anticipated  that  capturing  time  spent  in  each  category  will  show 
relationships  such  as  the  less  time  spent  planning,  the  more  time  and  expense  spent  in  the 
latter  categories.  Likewise,  developing  such  trends  will  also  assist  in  estimating  cost  and 
schedule  for  future  projects. 
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II.  CURRENT  ARDEC  PROCESSES,  PROPOSED  PROCESS 
STRUCTURE  AND  DESCRIPTION 

A.  CURRENT  ARDEC  PROCESSES 

There  is  no  institutionalized  methodology  used  to  conduct  operational  simulation 
and  analyses  within  the  OS&A  branch.  As  a  result,  new  projects  are  initiated  without 
consideration  to  past  lessons  learned.  This  is  highly  undesirable  since  aspects  of  previous 
analyses  may  apply  to  current  studies  resulting  in  a  significant  savings  of  cost  and  time. 
By  implementing  a  configuration  management  scheme  to  maintain  record  of  previous 
work  perfonned,  the  OS&A  branch  would  build  up  additional  background  data  to  further 
support  analytical  results  and  assist  the  Project  Lead  in  obtaining  results  more  quickly. 

Furthennore,  issues  such  as  striving  for  continuous  improvement  by  adapting  to 
best  business  practices  (BBPs),  heuristics,  and  current  tactics,  techniques,  and  procedures 
(TTPs)  have  not  been  adopted  consistently  nor  has  the  emphasis  of  capturing  the 
appropriate  problem  space  early  in  the  process. 

Finally,  projects  are  forced  into  cost,  schedule,  and  performance  overruns  from 
the  start  because  the  government  business  model  calls  for  time  and  schedule  estimates 
before  the  scope  of  the  problem  statement  and  project  requirements  are  methodically 
derived.  Issues  such  as  these  have  caused  the  branch  to  lose  credibility  in  the  eyes  of  the 
customer. 

An  example  of  one  project  which  suffered  due  to  lack  of  a  standardized  process 
was  the  Homeland  Defense  (HLD)  Project  (Przywozny  2010).  This  project  entailed 
simulating  an  attack  at  a  public  transportation  terminal  located  within  the  United  States. 
As  a  result  of  not  adopting  a  systems  engineering  process,  the  ARDEC  Project  Officer 
(APO)  aka  Project  Lead  was  given  a  massive  responsibility  to  not  only  execute  the  study 
without  the  discipline  of  a  systems  approach,  but  to  also  provide  the  direction,  control, 
and  oversight  to  all  aspects  of  the  study. 
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This  particular  project  encountered  issues  at  inception  when  the  customer  dictated 
the  modeling  and  simulation  application  solution  set  to  be  used.  Such  an  imposition 
proved  detrimental  throughout  the  execution  of  the  analysis.  The  problem  statement 
which  was  later  concluded  to  be  the  following: 

“ Analyze  the  force  effectiveness  of  ARDEC  products  within  a  given  scenario ,” 
became: 

“ Assess  the  utility  of  the  imposed  solution  set  for  analyzing  the  force  effectiveness 
of  ARDEC  products  within  a  given  scenario .” 

Upon  a  peer  review  of  the  project  after  its  completion,  one  issue  was  that  no 
process  which  resulted  in  the  generation  of  a  methodically  researched  problem  statement 
had  been  executed.  Consequentially,  no  formal  agreement  had  been  made  between  the 
customer  and  project  lead  indicating  a  common  understanding  of  the  project  and  problem 
statement  at  hand. 

Furthermore,  as  a  result  of  imposing  a  solution  prior  to  fully  understanding  the 
problem  itself,  a  fundamental  axiom  to  the  very  essence  of  implementing  the  systems 
approach  (Avoid  premature  adoption  of  ‘the  solution’)  was  violated: 

Succinctly  stated,  “The  design  team  should  be  careful  to  avoid  early  adoption  of  a 
candidate  system  from  a  previous  mission  in  order  to  avoid  being  locked  into  a  system 
that  only  marginally  meets  or  does  not  meet  your  objectives/requirements.”  (National 
Council  of  Space  Grant  Directors  2012) 

Finally,  the  timeline  and  cost  estimates  were  severely  underestimated.  This 
occurred  because  clarifications  of  the  customer  objective  and  upfront  statements  of  risks, 
issues,  and  constraints  were  not  identified.  Had  the  incorporation  of  a  systems  approach 
such  as  the  process  proposed  been  in  place  and  exercised,  it  is  likely  that  the 
determination  that  the  scope  of  the  scenario  coupled  with  the  constraints  dictated  by  the 
customer  was  beyond  the  Project  Lead’s  grasp  given  the  amount  resources  available. 
Additionally,  since  there  had  been  no  configuration  management  practice  in  place,  there 
was  no  previous  reference  with  which  to  draw  upon  in  order  to  detennine  if  a  similar 
project  such  as  this  had  been  exercised  before. 
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B. 


PROPOSED  PROCESS  STRUCTURE 


1.  Overview 

The  proposed  process  has  been  abstracted  into  four  key  phases.  Within  each 
phase,  steps  and  sub-steps  are  outlined.  The  overall  architecture  provides  general 
guidance  and  allows  the  Project  Lead  to  adapt  as  applicable  to  their  specific  project.  The 
phases:  Initiation,  Planning,  Execution,  and  Analysis  represent  major  categories  of 
project  progression  and  align  with  the  DoD  5000  Defense  Acquisition  System  Life  Cycle 
Model  as  illustrated  in  Figure  4  (Defense  Acquisition  University  2011). 
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Figure  4.  Process  Alignment  with  DoD  Defense  Acquisition  System  Life  Cycle 
Framework  (From:  Defense  Acquisition  University  2011) 


The  encapsulated  steps  are  designed  to  support  the  current  phase,  iterative  by 
nature,  and  sequential.  Typically,  resultant  data  from  one  step  becomes  input  for  the  next. 
A  branch  hierarchy  and  definition  of  personnel  roles  for  each  step  are  also  defined  in 
order  to  assist  the  Project  Lead  in  developing  a  project  schedule  and  budget  reliably  and 
apply  the  technique  of  resource  balancing.  Potential  implications  resulting  from  changes 
to  DoD  life  cycle  policies  serve  to  emphasize  the  fluidity  of  the  proposed  process.  While 
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the  process  is  not  required  to  map  to  the  DoD  5000  life  cycle,  the  intent  of  the 
comparison  is  to  provide  the  Project  Lead  an  element  of  confidence  and  familiarity  when 
adopting  this  process. 

2.  Branch  Hierarchy  and  Personnel  Roles 

Figure  5  illustrates  the  Operational  Simulation  &  Analysis  (OS&A)  branch 
structure  and  key  player  hierarchy.  This  hierarchy  was  created  through  the  OS&A 
branch’s  collective  vision  of  what  the  structure  of  a  balanced  analytical  team  should  look 
like  and  was  further  refined  when  put  into  practice  This  hierarchy  serves  as  the  primary 
resource  for  the  personnel  roles  defined  in  the  following  table. 


Figure  5. 


Typical  Operational  Simulation  &  Analysis  Branch  Personnel  /  Role 
Structure  (From:  Luzzi  2010) 
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Table  1  presents  (in  alphabetical  order)  descriptions  of  the  roles  introduced  in 
Figure  5.  Additionally,  the  roles  of  customer,  network  administrator,  and  stakeholder 
have  been  added.  While  these  roles  are  not  illustrated  in  the  figure  above,  this 
discrepancy  serves  a  benefit  by  showing  that  these  processes  and  structures  are  indeed 
living  and  will  likely  be  incorporated  into  future  versions  of  the  OS&A  branch  hierarchy. 


Role _ Description 


Battlemaster 

Individual  who  maintains  control  of  scenario  execution 

Blue  Player  / 
Commander  /  Team 

Any  entity;  human  or  machine;  which  acts  on  the 
‘friendly’  side  of  the  scenario 

Customer 

Any  individual  or  organization  who  places  a  vested 
interest  in  the  project 

Military  Advisor 
(Lead) 

Individual  who  serves  as  the  scenario  subject  matter 
expert  ensuring  that  the  scenario  defined  for  the  project  is 

relevant 

Network 

Administrator 

Individual  who  maintains  and  implements  the  necessary 
communications  and  routing  as  required  by  and  best 
suited  for  the  project’s  needs 

Individual  responsible  for  defining  what  data  needs  to  be 

coiiected  in  order  to  produce  the  requested  anaiyses,  as 
Project  Analyst  /  Lead  well  as  all  mathematical  and  statistical  calculations 
Analyst  necessary  to  derive  results.  In  most  cases,  the  project 


analyst  will  have  specific  subject  matter  expertise  in 

ORSA 

Project  Lead 

Individual  who  serves  as  the  customer’s  focal  point  and 
manages  project  cost  and  schedule 

Red  Player  / 
Commander  /  Team 

Any  entity,  human  or  machine,  which  acts  on  the  ‘enemy’ 
side  of  the  scenario 

Software  Engineer  / 
Lead  /  Developers 

Individual  who  develops,  modifies,  acquires  or  configures 
software  for  the  project 

Anyone  who  has  exposure  to  affeets  or  is  affected  by  the 

Stakeholder 

analyses  conducted. 

Supervisor  /  OS&A 
branch  Chief 

Individual  who  facilitates  the  Project  Lead  and  commits  to 
assuring  provision  of  assets  funding  and  time 

Systems  Engineer 

Individual  responsible  for  ensuring  that  all  aspects  of  the 
project’s  System’s  Engineering  functions  are  being 
executed 

Table  1. 

Process  Key  Players  and  Descriptions 
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c. 


PROCESS  DESCRIPTION 


The  remainder  of  this  thesis  shall  illustrate  the  proposed  process  using  the 
following  structure: 

•  A  flowchart  figure  depicting  current  phase,  step,  or  sub-step. 

•  A  table  indicating  key  players,  inputs,  and  outputs  for  the  current  phase, 
step,  or  sub-step 

•  A  discussion  of  current  phase,  step,  or  sub-step 

1.  Project  Initiation  Phase 

The  Project  Initiation  phase  (Figure  6)  consists  of  six  steps  which  are  intended  to 
provide  the  Project  Lead  with  a  precursory  view  of  the  problem  space.  Moreover,  it  is 
important  to  make  the  distinction  that  this  phase  is  targeted  toward  establishing  the 
project  scope  than  to  the  application  of  the  systems  approach  itself.  The  systems  approach 
is  implemented  in  the  planning  phase  as  will  be  discussed  in  Section  2.  The  only  thing 
assumed  about  the  project  at  this  time  is  that  there  has  been  a  stated  analytical  need. 


Figure  6.  Initiation  Phase  of  the  Proposed  Process 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Project  Analyst 
Developers 

Need  for  Analysis  to  be 
conducted 

Initial  (Retainer)  Funding 

Table  2.  Key  Players,  Inputs,  and  Outputs  for  the  Project  Initiation  Phase 
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a.  Gather  Information 

The  purpose  of  the  Gather  Information  step  is  to  elicit  responses  from  the 
customer  in  order  to  gain  initial  insight  to  the  problem  at  hand.  Activities  include 
customer  and  stakeholder  identification  and  interview.  The  Project  Lead  and  Systems 
Engineer  gather  information  to  identify  the  project  scope,  customer’s  anticipated  timeline 
for  project  initiation  and  completion,  the  general  level  of  financial  investment  on  the 
customer’s  behalf,  and  initial  concerns  that  could  negatively  affect  the  outcome  of  the 
project. 


Initiation 

Begin 


1.1  Gather 
Information 


1.2  Develop 
Preliminary  Schedule 


Y" 


1.3  Document  Risks, 
Issues,  and 
Constraints 


Wm 


'V' 


1.4  Configuration 
Management 


wm 


W 


1.5  Obtain  Customer 
Approval 


M 


1.6  Obtain  Funding 


Initiation 

End 


Figure  7. 


Gather  Information  Step  within  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Need  for  Analysis  to  be 
conducted 

General  Information 

Table  3.  Key  Players,  Inputs,  and  Outputs  for  the  Gather  Information  Step 


In  doing  so,  the  Project  Lead  and  Systems  Engineer  develop  an  initial 
insight  into  the  analysis  requested  and  the  initial  constraints  within  which  the  project  is 
bound.  Failure  to  adequately  conduct  this  portion  of  the  information  gathering  process 
can  lead  to  misguided  customer  and  stakeholder  identification,  erroneous  impression  of 
common  and  disjoint  visions,  and  an  overall  poor  initial  understanding  of  the  problem 
space  which  may  result  in  cost,  schedule,  and  perfonnance  overruns. 

(1)  Identify  Customers.  A  customer  is  a  “user,  operator,  and  integrator  of 
operational  products  at  any  position  within  a  system  structure”  (Defense  Acquisition 
University  2011).  In  projects  that  have  multiple  customers,  it  is  the  responsibility  of  the 
Project  Lead  and  Systems  Engineer  to  ensure  the  final  product  realizes  the  collective 
vision  of  these  customers. 
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Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

People  interested  in  the 
project 

Customer  List 

Table  4.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  Customers  Step 


(2)  Identify  Stakeholders.  A  stakeholder  is  an  “Individual  or  organization 
having  a  right,  share,  claim,  or  interest  in  a  system  or  in  its  possession  of  characteristics 
that  meet  their  needs  and  expectations”  (Stevens  Institute  of  Technology  2011). 
Furthermore,  “Stakeholders  include,  but  are  not  limited  to  end  users,  end  user 
organizations,  supporters,  developers,  producers,  trainers,  maintainers,  disposers, 
acquirers,  customers,  operators,  supplier  organizations  and  regulatory  bodies”  (Stevens 
Institute  of  Technology  2011). 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

People  interested  in  the 
project 

Stakeholder  List 

Table  5.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  Stakeholders  Step 


(3)  Conduct  Interviews.  Interviewing  customers  and  stakeholders  is 
the  key  to  obtaining  a  grasp  as  to  the  scope  of  the  problem  space.  It  bears  reiteration  that 
the  purpose  of  conducting  interviews  during  this  phase  relates  more  to  identifying  who 
the  players  are,  what  their  understanding  of  the  perceived  need  is,  and  gain  a  very  high 
level  impression  of  scope  of  the  project.  The  interviewing  process  begins  with  gathering 
notes  on  each  customer’s  perceived  needs,  thus  deriving  customer  expectations  prior  to 
any  systems  approach  implementation. 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Project  Analyst 

Customer  and  Stakeholder 
identification 

Top  Level  Requirements 
and  Expectations,  System 
ConOps 

Table  6.  Key  Players,  Inputs,  and  Outputs  for  the  Conduct  Interviews  Step 
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The  Systems  Engineer,  in  conjunction  with  the  Project  Lead  and  Project 
Analyst  take  the  time  to  interview  stakeholder  and  customers  with  the  goal  of 
ascertaining  the  following  information: 

•  Do  the  customers  know  what  they  want? 

•  Do  the  customers  have  the  same  goals? 

•  What  is  the  level  of  effort  of  this  project? 

•  Who  needs  to  be  involved? 

•  What  is  the  timeline  from  project  start  to  completion? 

•  What  is  the  financial  investment  of  the  funding  organization? 

It  is  important  to  note  that  the  interviews  conducted  provide  a  general 
picture  of  what  is  desired  but  do  not  constitute  the  “Requirements  Elicitation”  process 
which  occurs  in  the  following  phase.  The  purpose  of  this  step  is  to  understand  a 
component  unique  to  the  field  of  operational  analyses-scenario  aggregation. 

The  size  and  aggregation  of  a  scenario  is  directly  proportional  to  the  level 
of  effort  involved.  Executing  analyses  which  assess  the  perfonnance  of  a  single  item 
requires  considerably  less  effort  than  executing  analyses  which  assess  the  effect  an  item 
has  on  the  overall  outcome  of  a  large  scale  engagement.  For  example,  the  level  of  effort 
involved  in  analyzing  the  probability  of  acquisition  of  an  acoustic  sensor  through 
modeling  and  simulation  is  less  than  analyzing  how  that  same  acoustic  sensor  affects  the 
entire  outcome  of  an  engagement. 

As  such,  this  step  establishes  top  level  perceptions  and  expectations  of  the 
customers  and  stakeholders  and  forms  a  precursory  concept  of  operations.  The  concept  of 
operations,  or  ConOps,  is  defined  to  “describe  how  the  system  will  be  operated  during 
the  life-cycle  phases  to  meet  stakeholder  expectations.  It  describes  the  system 
characteristics  from  an  operational  perspective  and  helps  facilitate  an  understanding  of 
the  system  goals”  (NASA  2007). 
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b.  Develop  Preliminary  Schedule 


Projections  that  address  potential  resources  needed  should  be  made  and 
updated  as  necessary.  After  initially  identifying  resources  by  job  description  or  by 
naming  a  specific  individual,  the  Systems  Engineer  and  Project  Lead  can  establish  a 
baseline  schedule.  Most  commonly  used  is  the  work  breakdown  structure  (WBS).  The 
WBS  is,  “a  product-oriented  family  tree  composed  of  hardware,  software,  services,  data, 
and  facilities  . . .  results  from  Systems  Engineering  efforts”  (Defense  Acquisition 
University  2011).  This  schedule  is  understood  to  be  preliminary  and  is  likely  to  be 
updated  frequently. 
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Figure  8. 


Develop  Preliminary  Schedule  Step  of  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Interview  Notes 

Preliminary  Schedule 

Table  7.  Key  Players,  Inputs,  and  Outputs  for  the  Develop  Preliminary 

Schedule  Step 


c.  Document  Risks,  Issues,  and  Constraints 

Risk  is  defined  as  the  “measure  of  the  inability  to  achieve  overall  program 
objectives  within  defined  cost,  schedule,  and  technical  constraints  and  has  two 
components:  (1)  the  probability  of  failing  to  achieve  a  particular  outcome  and  (2)  the 
consequences  /  impacts  of  failing  to  achieve  that  outcome.”  (NASA  2007). 

The  goal  of  this  step  is  to  clearly  identify  potential  project  pitfalls  and  to 
ultimately  be  proactive  in  addressing  them  as  opposed  to  reacting  to  them  once  they 
occur.  Furthermore,  continual  consideration  of  risks,  issues,  and  constraints  give  the 
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customer  a  realistic  view  and  help  ground  their  expectations.  Documenting  risks,  issues, 
and  constraints  provide  an  element  of  protection  to  the  Project  Lead  such  that  any 
potential  pitfall  that  may  be  encountered  is  documented,  discussed,  and  the  remedies  and 
mitigations  set  forth  are  agreeable  to  all  parties  involved. 
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Figure  9. 


Document  Risks,  Issues,  and  Constraints  Step  in  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Interview  Notes, 
Preliminary  Schedule 

Open  statements  of  concern 
for  stakeholders  to  consider 

Table  8.  Key  Players,  Inputs,  and  Outputs  for  the  Document  Risks,  Issues,  and  Constraints 

Step 


It  is  critical  that  the  Risks,  Issues,  and  Constraints  are  monitored 
continually,  as  the  project  progresses,  the  status  of  these  items  will  most  assuredly 
change.  Within  the  process,  just  as  in  the  Defense  Acquisition  University’s  discipline  of 
System’s  Acquisition,  risk  identification  will  be  categorized  as  low,  moderate,  and  high 
based  the  likelihood  of  the  event  occurring  and  severity  of  occurrence. 
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The  following  figure  depicts  a  typical  risk  matrix: 


1 


Consequences 


Figure  10.  Matrix  Used  to  Assess  Level  of  Risk  Based  on  the  Likelihood  of 
Occurrence  Cross  Referenced  with  the  Severity  of  the  Consequence 

(From:  NASA  2007) 


For  all  identified  risks,  a  mitigation  approach  is  identified.  Additionally, 
risk  concerns  require  multiple  iterations  throughout  the  life  cycle  of  the  project.  Since  the 
process  does  not  change,  future  occurrences  of  the  “Document  Risks,  Issues,  and 
Constraints”  step  are  referred  back  to  this  section. 
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Table  9  summarizes  the  Continuous  Risk  Management  (CRM)  process 
employed  and  referenced  in  the  NASA  Systems  Engineering  Handbook: 


Step 

Explanation 

Identify 

Statement  of  risk,  likelihood  and  scenario  with  which  it  may 

occur 

Analyze 

Estimate  likelihood  and  severity  of  risk 

Plan 

Establish  method  to  monitor  established  risk 

Track 

Maintain  current  status  of  each  risk,  including  new 

observations,  and  resolutions 

Control 

Execution  of  plan 

Communicate  & 
Document 

Living  record  of  risks  identified 

Table  9.  NASA  Continuous  Risk  Management  (CRM)  Process  (From:  NASA  2007) 


Issues  are  immediate  problems  which  must  be  solved  (Best-Practice.com 
2012).  The  failure  to  address  risks  within  their  appropriate  scope  will  result  in  issues.  One 
example  of  an  issue  encountered  during  the  HLD  Case  Study  (described  in  Chapter  II, 
Section  B)  was  the  determination  of  the  desired  scenario  size.  In  order  to  model  the 
scenario  realistically,  developers  needed  to  create  thousands  of  entities  using  a  3-D 
modeling  application  such  as  Presagis’  Creator©,  DI-Guy©,  or  PTC’s  Pro-Engineer©. 
Entities  requiring  simulation  ranged  from  human  actors,  to  high  resolution  vehicles, 
buildings,  and  explosive  ordnance.  Development  of  such  entities  requires  a  significant 
time  investment  based  on  fidelity  (also  further  emphasizing  the  importance  of 
configuration  management;  reduce,  reuse,  and  recycle).  During  the  interview  process,  the 
scenario  size  quickly  became  recognized  as  a  risk.  However,  this  risk  was  not  identified 
the  costumer  and  as  the  project  progressed,  it  quickly  resulted  in  an  issue.  Failure  to 
reduce  the  scope  of  the  scenario  in  order  to  create  a  manageable  scenario  size  resulted  in 
an  issue  during  the  scenario  generation  process.  In  turn,  the  project  suffered  in  terms  of 
schedule  since  the  scenario  had  to  be  reduced  at  such  a  late  point  in  the  project. 

Finally,  constraints,  as  opposed  to  issues,  are  limitations  imposed  upon  the 
project  by  some  means;  including  but  not  limited  to  requirements,  technological 
limitations,  timelines  and  personnel. 
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d.  Configuration  Management 


The  NASA  Systems  Engineering  handbook  defines  Configuration 
Management  as, 

a  management  discipline  applied  over  the  product’s  life  cycle  to  provide 
visibility  into  and  to  control  changes  to  performance  and  functional  and 
physical  characteristics.  Configuration  Management  ensures  that  the 
configuration  of  a  product  is  known  and  reflected  in  product  information, 
that  any  product  change  is  beneficial  and  is  effected  without  adverse 
consequences,  and  that  changes  are  managed.  (NASA  2007) 
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Figure  1 1 .  Configuration  Management  Step  of  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Interview  Notes, 

Project  Fead 

Preliminary  Schedule, 

Controlled  method  to 

Developers 

Risks,  Issues  and 

preserve  data 

Constraints 

Table  10.  Key  Players,  Inputs,  and  Outputs  for  the  Configuration  Management  Step 


Furthennore,  use  of  configuration  management  “reduces  technical  risks  by 
ensuring  correct  product  configurations,  distinguishes  among  product  versions,  ensures 
consistency  between  the  product  and  information  about  the  product,  and  avoids  the 
embarrassment  of  stakeholder  dissatisfaction  and  complaint”  (NASA  2007). 

Use  of  a  configuration  management  system  is  of  the  utmost  importance. 
By  employing  a  configuration  management  very  early  in  the  process,  the  Systems 
Engineer  and  project  employees  can  document  where  diversions  are  made,  and  people 
working  on  future  projects  can  refer  back  to  this  asset  library  to  see  what  can  be  reused, 
thus  creating  a  major  benefit  in  terms  of  time,  cost,  and  schedule  reduction. 
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The  configuration  management  library  is  the  primary  hub  in  which  all 
recorded  data  is  maintained  for  the  life  of  the  project  and  beyond.  All  artifacts  from  the 
earliest  to  the  latest  phase  of  the  cycle  should  be  contained  within  this  library  as  it  is 
elemental  for  providing  justifications  and  reiterations  of  actions  and  reactions  taken.  Use 
of  a  configuration  management  policy  protects  the  Project  Lead  and  workers  such  that  it 
is  used  to  provide  a  running  record  for  activities.  Also  of  great  importance  is  the  ability  to 
look  back  over  specific  time  periods  to  detennine  what  activities  were  occurring  at  that 
time  and  why.  Requirements  which  have  changed  are  captured  through  this  mechanism 
and  provide  a  record  for  use  in  future  projects  which  may  reuse  parts  of  the  data  or 
scenario  recorded. 

Table  1 1  provides  an  excellent  structure  for  the  configuration  management 
of  requirements.  This  table  is  located  in  chapter  4  of  the  NASA  Systems  Engineering 
handbook. 


Item 

Function 

Requirement  ID 

Unique  numbering  system  for  sorting  and  tracking 

Rationale 

Additional  infonnation  which  clarifies  the  requirement  intent  at 

time  of  writing 

Traced  From 

Capture  bi-directional  traceability  from  parent  to  child 
requirements 

Owner 

Party  responsible  for  achieving,  managing,  and/or  approving 
changes  to  requirement 

Verification  Method 

Captures  method  of  verification  (test,  inspection,  analysis, 
demonstration) 

Verification  Lead 

Person  responsible  for  verifying  the  requirement 

Verification  Level 

Specifies  level  in  which  requirement  will  be  verified  (system, 
subsystem,  component,  etc.) 

Table  11.  NASA  Requirements  Meta-Data  (From:  NASA  2007) 


Since  the  process  does  not  change  future  occurrences  of  the  Configuration 
Management  step  are  referred  back  to  this  section. 
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e.  Obtain  Customer  Approval 

Customer  approval  at  this  juncture  indicates  that  the  Project  Lead  and 
Systems  Engineer  have  interpreted  the  problem  correctly  and  that  there  has  been 
sufficient  communication  with  the  customer  to  derive  revised  and  updated  goals  and 
necessary  plans.  The  approval  obtained  indicates  that  the  customer  agrees  to  the  problem 
definition  and  proposed  manner  of  project  execution. 
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Figure  12. 


Obtain  Customer  Approval  Step  of  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Configuration  Managed 
Data 

Customer  Reaction 

Table  12.  Key  Players,  Inputs,  and  Outputs  for  the  Obtain  Customer  Approval  Step 


f  Obtain  Funding 

This  step  requires  that  funding  be  established  as  a  standard  amount  in 
order  for  the  Project  Lead  and  designated  team  members  to  receive  compensation  for 
their  efforts.  In  support  of  implementing  this  ‘retainer’  funding  concept,  consider  the 
government  acquisition  process  where  one  of  the  preconditions  to  achieving  Milestone  B 
is  having  secured  funding  for  the  remainder  of  the  project.  Such  a  stipulation  implies  that 
other  activities  previous  to  Milestone  B  which  were  accomplished  through  a  different  set 
of  funds. 
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Figure  13. 


Obtain  Funding  Step  of  the  Initiation  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Customer  Reaction 

Customer  Funding 

Table  13.  Key  Players,  Inputs,  and  Outputs  for  the  Obtain  Funding  Step 


The  process  proposed  asks  for  the  same  separation  of  funding  however;  it 
is  a  highly  debatable  notion  which  does  not  currently  exist  within  the  government’s 
business  model  for  perfonning  analyses.  Nevertheless,  it  is  imperative  to  point  out  that 
this  has  historically  been  a  critical  failure  area.  An  unacceptable  amount  of  risk  is  often 
assumed  when  costs  and  timelines  are  determined  prior  to  proper  breakdown  and  analysis 
of  system  requirements. 

Early  decisions  in  the  Systems  Engineering  process  tend  to  have  the 
greatest  effect  on  the  resultant  system  life-cycle  cost”  (NASA  2007). 
“Typically,  by  the  time  the  preferred  system  architecture  is  selected, 
between  50  and  70  percent  of  the  system’s  life-cycle  cost  has  been  locked 
in”  (NASA  2007).  “By  the  time  a  preliminary  system  design  is  selected, 
this  figure  may  be  as  high  as  90  percent.  This  presents  a  major  dilemma  to 
the  Systems  Engineer,  who  must  lead  this  selection  process.  Just  at  the 
time  when  decisions  are  most  critical,  the  state  of  information  about  the 
alternatives  is  least  certain.  (NASA  2007). 
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2.  Planning  Phase 


Historically,  the  key  in  defining  success  for  most  projects  lie  in  a  significant 
investment  of  personnel  in  the  planning  phase.  The  action  of  planning,  which  is  discussed 
throughout  this  document  is  generally  underestimated.  When  this  occurs,  executing 
projects  suffer  in  terms  or  cost,  schedule,  perfonnance,  or  some  combination  of  all  three. 
In  addition,  lack  of  thorough  planning,  either  from  lack  of  time  or  lack  of  resources, 
ultimately  contributes  to  the  reputation  of  the  branch.  Maintaining  a  reputation  of  high 
standards  is  critical  to  mission  success  and  sustenance  of  the  division.  In  general,  one  can 
estimate  that  the  steps  detailed  in  the  process’s  Planning  Phase  will  consume  up  to  half  of 
the  project’s  lifetime,  though  probably  not  so  much  of  the  budget  since  there  are  fewer 
resources  needed  in  this  phase  than  in  others.  This  phase  consists  of  all  activities 
necessary  to  determine  the  customer’s  requirements  and  devise  a  mechanism  which 
exercises  said  requirements. 


Figure  14.  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Project  Analyst 
Military  Advisor 
Software  Engineer  & 
Developers 

Stakeholder  and  Customer 
Identification 

Problem  Statement 
Requirements 

Metrics 

Scenario 

Data  Collection  Plan 

Table  14.  Key  Players,  Inputs,  and  Outputs  for  the  Planning  Phase 
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a. 


Requirements  Generation 


The  requirements  generation  step  is  arguably  the  most  important  predictor 
as  to  the  outcome  of  the  project.  Well  defined  requirements  lead  to  well  defined 
objectives  for  analyses  while  poorly  defined  requirements  lead  to  confusion.  Technical 
requirements  are,  “the  approved  set  of  requirements  that  represents  a  complete 
description  of  the  problem  to  be  solved  and  requirements  that  have  been  validated  and 
approved  by  the  customer  and  stakeholders”  (NASA  2007). 
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Figure  15.  Requirements  Generation  Step  of  the  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Identified  Customers  and 
Stakeholders 

High  Level  Requirements 

Table  15.  Key  Players,  Inputs,  and  Outputs  for  the  Requirements  Generation  Step 


(1)  Establish  the  Problem  Statement.  After  initial  and  follow  up 
interviews,  the  Systems  Engineer  should  have  a  fairly  clear  picture  of  the  customer 
hierarchy  and  dynamic.  The  problem  statement,  which  is  a  “brief,  concise  statement  of 
fact  that  clearly  describes  an  undesirable  state  or  condition  without  identifying  the  source 
or  actions  required  to  solve  the  problem”  (Wasson  2005),  should  be  as  concise  as 
possible  without  being  contrived.  There  is  a  delicate  balance  between  making  the 
problem  statement  too  short,  such  that  elements  are  left  for  interpretation,  and  too  long 
such  that  the  scope  of  the  project  is  far 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

High  Level  Requirements 

Problem  Statement 

Table  16.  Key  Players,  Inputs,  and  Outputs  for  the  Establish  the  Problem  Statement 

Sub-Step 
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After  initial  and  follow  up  interviews,  the  Systems  Engineer  should  have  a 
fairly  clear  picture  of  the  customer  hierarchy  and  dynamic.  The  problem  statement,  which 
is  a  “brief,  concise  statement  of  fact  that  clearly  describes  an  undesirable  state  or 
condition  without  identifying  the  source  or  actions  required  to  solve  the  problem” 
(Wasson  2005),  should  be  as  concise  as  possible  without  being  contrived.  There  is  a 
delicate  balance  between  making  the  problem  statement  too  short,  such  that  elements  are 
left  for  interpretation,  and  too  long  such  that  the  scope  of  the  project  is  far  too  daunting 
an  undertaking.  In  general,  the  problem  statement  should  span  from  a  simple  sentence  to 
a  few  sentences  forming  no  more  than  a  brief  paragraph. 

When  deriving  the  problem  statement,  it  is  best  to  keep  the  following 
practices  in  mind: 

•  Do  not  identify  the  cause  of  the  problem;  simply  state  what  the  problem  is 

•  Do  provide  the  environment  which  precipitates  the  problem 

•  Do  not  establish  any  explicit  or  implicit  solutions  (Wasson  2005) 

(2)  Decompose  the  Problem  Statement.  Decomposition  of  the  problem 
statement  or  partitioning  the  problem  space  is  the  first  step  to  grasping  the  complete 
context  of  the  issue  at  hand.  Decomposition,  or  partitioning,  provides  the  ability  to, 
“isolate  key  properties  and  characteristics  of  the  problem  as  abstractions  that  enable  us  to 
ultimately  develop  solutions”  (Wasson  2005).  The  problem  statement  decomposition  has 
the  following  characteristics. 

•  Components  are  distinct 

•  Functions  are  not  redundant 

•  Interfaces  between  components  are  clear 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 

Problem  Statement 

Problem  Statement 
Decomposition 

Table  17.  Key  Players,  Inputs,  and  Outputs  for  the  Decompose  the  Problem  Statement  Sub- 

Step 
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It  is  from  the  problem  statement  decomposition  that  we  can  impose  some 
order,  or  outline  and  clearly  define  the  requirements. 

(3)  Identify  Requirements.  Requirements  need  to  be  stated  in 
unambiguous  terms.  Requirements  derivation  is,  “The  act  of  decomposing  an  abstract 
parent  requirement  into  lower  level  objective,  perfonnance-based  sibling  actions. 
Collective  accomplishment  of  the  set  of  derived  “sibling”  actions  constitutes  satisfactory 
accomplishment  of  the  “parent”  requirement”  (Wasson  2005). 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

Interview  Notes 

High  Level  Requirements 
Problem  Statement 
Problem  Statement 
Decomposition 

Detailed  Requirements 

Table  18.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  Requirements  Sub-Step 


Well  stated  requirements  are  explicit  and  demonstrate  minimal  risk  for 
misinterpretation.  Additionally,  requirement  relevancy  is  assured  through  a  traceability 
process  which  ensures  requirements  are  complete.  Furthennore,  requirements  each  have  a 
subjective  value  and  priority  to  the  user,  impose  constraints  on  design  solution  options, 
potentially  increase  risk,  and  have  a  cost  associated  with  them  for  implementation  and 
maintenance. 

Requirements  should  be: 

•  unique  within  the  system, 

•  singular  in  purpose, 

•  consistent  with  other  requirements 

•  non-conflicting 

•  explicitly  realistic,  achievable,  consistent,  testable,  measurable, 

and  verifiable 

•  assigned  to  an  owner  who  is  accountable  for  its  implementation 
and  maintenance  (NASA  2007,  Wasson  2005) 
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Finally,  a  requirement  is  considered  finalized  if  and  only  if  it  is  traceable 
to  the  problem  statement,  has  an  established  verification  method  (generally  some  level  or 
type  of  testing),  and  is  agreed  upon  by  the  customers/stakeholders.  Well  defined 
requirements  establish  the  basis  for  agreement  between  the  stakeholders  and  the  Project 
Lead  on  what  the  project  is  to  accomplish.  They  provide  a  basis  for  reliably  estimating 
costs  and  schedules,  and  provide  a  verification  and  validation  baseline  (NASA  2007). 

(4)  Create  Evaluation  Metrics.  Once  requirements  have  been  established  in 
quantifiable  terms,  the  Systems  Engineer  must  attempt  to  apply  values  and  thresholds  in 
order  to  establish  whether  or  not  the  requirement  has  indeed  been  met.  Additionally,  the 
Systems  Engineer  should  have  a  good  idea  of  the  relative  importance  of  the  requirements, 
as  some  are  more  critical  than  others.  The  weighting  of  these  requirements  will  ultimately 
factor  into  any  trade  off  comparison. 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
Project  Lead 

System  Analyst 

Detailed  Requirements 

Evaluation  Metrics 

Table  19.  Key  Players,  Inputs,  and  Outputs  for  the  Create  Evaluation  Metrics  Sub-Step 


Measures  of  evaluation,  or  Technical  Performance  Measures  are,  “An 
established  set  of  measures  based  on  the  expectations  and  requirements  that  will  be 
tracked  and  assessed  to  determine  overall  system  or  product  effectiveness  and  customer 
satisfaction”  (Wasson  2005).  Terms  used  for  these  evaluation  metrics  are  based  on  the 
type  of  analysis  being  performed.  The  level  of  aggregation  of  the  scenario  under  analysis 
typically  determines  which  measures  will  be  used.  In  the  OS&A  branch’s  case,  the  most 
commonly  used  measure  is  the  Measure  of  Effectiveness  (MOEs).  Figure  16,  from  the 
INCOSE  Technical  Measurement  Guide,  demonstrates  the  hierarchy  of  technical 
measures. 


28 


Technical  Measurement 


INCOSE-TP  -2003-020-01 
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Figure  16.  Relationship  of  the  Technical  Measures  (From:  Roedler  2005) 


(5)  Create  Functional  Decomposition.  The  functional  decomposition  is  the 
logical  compartmentalization  of  requirements  into  groups.  The  ability  to  separate  families 
of  requirements  into  chunks  allows  for  a  modular  approach.  This  is  an  important  step  to 
perform  as  it  also  helps  to  define  the  resources  needed  to  accomplish  any  task.  The 
functional  decomposition  is  intended  to  “translate  top-level  requirements  into  functions 
that  must  be  performed  to  accomplish  the  requirements.  Decompose  and  allocate  the 
functions  to  lower  levels  of  the  product  breakdown  structure.  Identify  and  describe 
functional  and  subsystem  interfaces”  (NASA  2007).  Finally,  the  resultant  set  of 
documents  composes  the  functional  baseline  for  the  system. 
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Key  Players 

Inputs 

Outputs 

Systems  Engineer 

Detailed  Requirements 

Functional  Decomposition 
Project  Scenario 
Specification  (System 
Specification) 

Table  20.  Key  Players,  Inputs,  and  Outputs  for  the  Create  Functional  Decomposition 

Sub-Step 


For  our  purposes,  the  output  of  this  process  is  referred  to  as  the  “Project 
Scenario  Specification.”  This  specification  serves  as  the  baseline  of  activities  which  the 
scenario  must  exercise.  The  Project  Scenario  Specification  should  be  thorough  enough 
such  that  there  is  sufficient  guidance  and  constraints  for  the  project  analyst  and  military 
advisor  to  develop  a  comprehensive  scenario  which  will  exercise  all  aspects  of  the 
requirements  set  forth. 

b.  Scenario  Generation 

The  Scenario  Generation  Step  defines  a  storyboard  which  exercises  each 
requirement  in  a  manner  consistent  with  its  evaluation  method. 
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Figure  17. 


Scenario  Generation  step  of  the  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 
System  Analyst 
Military  Advisor 

Detailed  Requirements 
Functional  Decomposition 

Scenario 

Data  Collection  Plan 

Table  21 .  Key  Players,  Inputs,  and  Outputs  for  the  Scenario  Generation  Step 
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(1)  Define  Scenario.  Given  the  requirements,  functional 
decomposition,  system  design  specification,  documented  risks  and  constraints  and 
configuration  management  scheme;  the  Project  Lead,  Project  Analyst,  and  Military 
Advisor  define  a  real  world  scenario  that  will  accurately  and  realistically  exercise  all 
requirements  set  forth.  Over  time,  it  is  likely  that  there  will  have  already  been  scenarios 
established  that  will  fit  the  existing  projects  needs  with  a  few  tweaks.  Reviewing  past 
projects  should  always  be  the  first  step  to  scenario  definition  and  as  a  result,  two 
outcomes  will  occur.  One  possibility  is  that  the  Systems  Engineer  will  find  an  appropriate 
scenario  given  some  adjustments.  In  this  case,  the  scenario  should  be  reviewed  with  a 
Military  Advisor  to  confirm  the  scenario’s  relevance  to  the  real  world.  The  second 
possibility  is  that  there  is  no  scenario  available  to  exercise  the  requirements  and  a  new 
scenario  must  be  created.  One  suggested  method  for  scenario  development  is  by  way  of  a 
logical/functional  architecture  development  methodology  or  logical  decomposition. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Requirements 

Storyboard  Scenario 
Interactions  Matrix 

Project  Analyst 

Functional  Decomposition 

Storyboard 

Military  Advisor 

Metrics 

Logical/Functional 

Architecture 

Table  22.  Key  Players,  Inputs,  and  Outputs  for  the  Define  Scenario  Sub-Step 


A  logical  decomposition  defines  the  ‘what’  which  must  be  achieved  by  the 
system  at  each  level  to  enable  a  successful  project  (NASA  2007).  The  functional 
decomposition  described  in  the  previous  step  above  is  an  element  of  logical 
decomposition.  This  process  enables  the  Project  Lead  and  Systems  Engineer  to 
thoroughly  understand  the  requirement  at  hand  and  to  break  it  down  into  logical 
components.  Inputs  to  decomposition  include  the  technical  requirements  and  measures. 

This  process  of  functional  and  logical  decomposition  enables  elements  of 
the  overall  system  to  be  developed  independently.  The  advantage  of  doing  so  is  evident 
in  terms  of  risk  and  time  reductions.  Tools  used  and  design  elements  produced  from  this 
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activity  include  work-breakdown  structure  (WBS),  Functional  Flow  Block  Diagrams 
(FFBD),  and  N2  Diagrams.  While  the  original  intent  for  these  standards,  processes,  and 
tools  are  for  the  development  and  manufacturing  of  physical  systems,  the  best  practices 
apply  and  are  also  in  alignment  with  the  development  of  Modeling  and  Simulation 
Analytical  systems. 

Functional  Flow  block  diagrams  depict  a  sequence  of  activities  or 
functions  which  are  derived  from  the  requirements  and  form  the  design  (NASA  2007). 
The  FFBD,  as  seen  in  Figure  18,  illustrates  sequential  as  well  as  parallel  activities.  In 
application  to  the  proposed  process,  an  FFBD  would  be  applied  to  the  scenario  generation 
section.  It  is  important  to  note  that  FFBDs  are  considered  high  level  because  they  define 
what  is  to  occur,  but  not  how  it  occurs. 


Figure  18.  Sample  Functional  Flow  Block  Diagram  (FFBD) 

N2  (or  N-squared)  diagrams,  as  shown  in  Figure  19,  define  functional 
interfaces  between  components  (NASA  2007).  The  combination  of  the  WBS,  FFBDs, 
and  N2  diagrams  provide  to  the  customer  a  hierarchical  functional  breakdown  of  the 
parent  requirements  (NASA  2007). 
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Figure  19.  Sample  N2  (N-squared)  Diagram 


As  per  the  Wasson  text,  the  logical  decomposition  process 
consists  of  the  following  steps: 

1 :  Identify  logical  objects  or  entities 

2:  Identify  each  entity’s  capabilities 

3:  Create  a  logical  interactions  matrix 

4:  Create  the  logical/functional  architecture  (Wasson  2005) 
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Conduct  Traceability  between  Requirements  and  Scenario.  As  the  scenario 
is  generated,  traceability  is  verified  via  a  document  or  spreadsheet  indicating  how 
scenario  elements  touch  upon  the  requirements.  The  scenario  must  exercise  all  of  the 
requirements.  If  this  is  not  the  case,  the  scenario  must  be  reworked  or  the  requirements 
revisited.  At  this  phase  in  the  life  cycle,  scenario  modification  will  likely  be  the  less 
expensive  of  the  two  options. 


Key  Players 

Inputs 

Outputs 

Systems  Engineer 

Detailed  Requirements 
Scenario 

Traceability  Matrix 

Table  23.  Key  Players,  Inputs,  and  Outputs  for  the  Conduct  Traceability  Sub-Step 


(2)  Identify  and  Resolve  Gaps.  The  act  of  Gap  Identification  and 
Resolution  closes  out  any  outstanding  items.  If  an  amenable  resolution  is  unattainable, 
the  Project  Lead  and  Systems  Engineer  must  discuss  this  with  the  customer  and 
stakeholders  then  backtrack  through  the  previous  steps  to  come  up  with  a  set  of  attainable 
requirements,  scenario,  or  both.  In  comparison  with  the  Systems  Engineering  handbook 
for  design  and  development  of  systems,  this  step  is  nearly  synonymous  with  the  Analysis 
of  Alternatives  step. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

System  Analyst 
Developers 

Gaps  Identified  through 
traceability  activity 

Solutions  that  will,  meet 
needs  as  stated,  modify 
requirements,  or  modify  the 
scenario. 

Table  24.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  and  Resolve  Gaps  Sub-Step 


(3)  Define  Data  Collection  Plan.  The  Data  Collection  plan  defines  all  the 
details  concerning  data  collection,  including  how  much  and  what  type  of  data  is  required 
and  when  and  how  it  should  be  collected.  Exercising  the  scenario  without  any  data 
collection  plan  and  mechanism  in  place  would  simply  yield  a  narrative  with  no  output  for 
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analytical  use;  thus,  a  great  expenditure  with  very  little  return  on  investment  (ROI). 
When  developing  the  plan,  consideration  of  the  scenario  and  the  requirements  are 
important.  Just  as  the  scenario  is  traced  back  to  the  requirements,  so,  too,  is  the  data 
collection  plan  to  the  scenario. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

System  Analyst 

Detailed  Requirements 
Metrics 

Scenario 

Data  Collection  Plan 

Table  25.  Key  Players,  Inputs,  and  Outputs  for  the  Define  Data  Collection  Plan  Sub-Step 


(4)  Verify  &  Validate  (V&V)  Scenario.  The  scenario,  once  established  and 
finalized,  must  go  through  a  final  review  to  ensure  to  the  customer  that  it  is  producible 
and  relevant. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

War  fighter  subject  matter 
expert  (SME) 

Detailed  Requirements 
Scenario 

Perceived  fitness  and  buy-in 
of  the  scenario  for  the 
project 

Table  26.  Key  Players,  Inputs,  and  Outputs  for  the  Verify  and  Validate  Scenario  Sub-Step 


c.  Define  Detailed  Schedule 

Given  a  well  defined  set  of  requirements  and  scenario  to  exercise,  the  next 
step  for  the  Project  Lead  and  Systems  Engineer  is  to  define  a  schedule  to  complete  the 
analysis.  Typically,  the  project  is  constrained  to  a  specific  time  limit,  and  it  is  the  job  of 
the  Project  Lead  and  Systems  Engineer  to  assess  whether  this  time  constraint  can  be  met. 
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Figure  20.  Define  Detailed  Schedule  Step  of  the  Planning  Phase 
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Key  Players 

Inputs 

Outputs 

Project  Lead 

Systems  Engineer 

Scenario 

Data  Collection  Plan 

Detailed  Schedule 

Table  27.  Key  Players,  Inputs,  and  Outputs  for  the  Define  Detailed  Schedule  Step 


d.  Define  Costs 

At  this  point  in  the  project’s  cycle,  the  Project  Lead  and  Systems  Engineer 
will  know  the  level  of  effort  needed  for  the  project.  It  is  important  to  consider  what  the 
customer  is  willing  to  spend  in  direct  contrast  to  what  the  team  is  able  to  accomplish. 
Should  the  costs  of  fulfilling  the  requirements  as  stated  exceed  the  budgeted  amount, 
revisiting  the  requirements  and  providing  a  reduction  in  scope  will  be  necessary. 
Conversely,  the  Project  Lead  may  opt  to  increase  the  fidelity  of  the  analysis  or  perform 
the  project  below  the  budgeted  amount  should  the  estimate  come  in  low. 


Figure  2 1 .  Define  Costs  Step  of  the  Planning  Phase 


2.2  Scenario 
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2.5  Document 
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and  Constraints 
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2.6 

Configuration 
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YV. _ JC 


2.7  Obtain 
Customer 
Approval 


2.8  Obtain 
Funding 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Detailed  Schedule 

Project  Budget 

Table  28.  Key  Players,  Inputs,  and  Outputs  for  the  Define  Costs  Step 


In  any  project,  one  must  consider  the  customer’s  priorities.  A  fairly 
popular  concept  illustrated  is  the  Project  Management  Triangle  (see  Figure  22),  which 
addresses  the  relationship  between  cost,  schedule,  and  performance  (or  quality).  Should 
the  project  lack  sufficient  funding,  then  the  Project  Lead  may  opt  to  lengthen  the  time  to 
complete.  Should  the  project  need  to  be  completed  within  a  small  time  frame,  one  can 
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expect  higher  costs  due  to  the  amount  of  manpower  assigned  to  meet  the  timeline.  Given 
the  nature  of  the  studies  performed,  the  “fast  and  cheap”  concept  is  not  an  option  for  our 
organization. 


Scope/Quality 


Figure  22.  The  Project  Management  Triangle  (From:  Wang  2010) 
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Figure  23. 


Document  Risks,  Issues  &  Constraints  Step  of  the  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Systems  Engineer 
System  Analyst 
Developers 

All  artifacts 

Updated  open  statements  of 
concern  for  stakeholders  to 
consider 

Table  29.  Key  Players,  Inputs,  and  Outputs  for  the  Document  Risks,  Issues  &  Constraints 

Step 
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f.  Configuration  Management 


As  in  the  previous  phases,  the  Project  Lead  in  conjunction  with  all 
involved  members  of  the  project  need  to  exercise  good  configuration  management 
practices  in  order  to  maintain  the  project  documentation  (living  documents)  for  storage 
and  future  use. 
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Figure  24. 


Configuration  Management  Step  of  the  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 
Developers 

All  existing  artifacts 

Controlled  method  to 
preserve  data 

Table  30.  Key  Players,  Inputs,  and  Outputs  for  the  Configuration  Management  Step 


g.  Obtain  Customer  Approval 

Upon  completion  of  the  preceding  steps  in  this  phase,  it  is  important  to 
review  activities  with  the  customer.  In  doing  so,  the  customer  gains  a  clear  view  of  what 
the  project  is  intended  to  do,  how  long  the  project  will  take,  and  is  aware  of  concerns  on 
behalf  of  the  analysis  team.  The  first  progress  review  represents  a  significant  milestone  in 
the  lifetime  of  the  project,  as  it  represents  a  go  /  no-go  moment.  Should  any  requirements 
be  inappropriate  or  incomplete,  correction  of  such  at  this  juncture  will  significantly 
reduce  any  overhead  or  rework  as  opposed  to  catching  and  fixing  later. 
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Figure  25.  Obtain  Customer  Approval  Step  of  the  Planning  Phase 
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Key  Players 

Inputs 

Outputs 

Customer 

Stakeholder 

Project  Lead 

All  existing  artifacts  to  date 

Customer  consent  or  need 
for  rework. 

Systems  Engineer 

Lead  Analyst 

Table  3 1 .  Key  Players,  Inputs,  and  Outputs  for  the  Obtain  Customer  Approval  Step 


h.  Obtain  Funding 

Upon  completion  of  requirements,  metrics,  scenario  definition,  and 
traceability  verification,  which  are  all  precursors  to  the  actual  execution,  the  remaining 
project  funding  should  be  in  place.  In  the  development  of  systems,  this  would  mark 
Milestone  B  in  product  development.  In  the  funding  of  systems,  it  is  at  this  point  in  time 
that  complete  project  funding  has  been  secured. 
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Figure  26.  Obtain  Funding  Step  of  the  Planning  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Detailed  Project  Schedule 
Project  Budget 

Customer  Funding 

Table  32.  Key  Players,  Inputs,  and  Outputs  for  the  Obtain  Funding  Step 


3.  Execution  Phase 

The  execution  phase  consists  of  steps  which  define  the  computing  environment, 
acquire  necessary  assets  (hardware,  software,  resources)  to  run  the  scenario,  develop 
procedures  for  scenario  execution,  and  finally  review  with  the  customer  the  scenario  and 
execution  at  hand.  At  this  point,  analysis  is  yet  to  be  perfonned. 
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Figure  27.  Execution  Phase 
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Table  33.  Key  Players,  Inputs,  and  Outputs  for  the  Execution  Phase 


a.  System  Realization 

The  system  realization  step  takes  all  work  previously  performed  and 
begins  to  place  it  in  action.  This  step  includes  identification  of  hardware,  software,  and 
network  solutions  and  well  as  personnel  resources. 
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Figure  28.  System  Realization  Step  of  the  Execution  Phase 
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Developers 

Project  Lead 

Project  Analyst 
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Data  Collection  Plan 

Physical  System 
Personnel  Assignments 

Table  34.  Key  Players,  Inputs,  and  Outputs  for  the  System  Realization  Step 
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(1)  Identify  Hardware,  Software,  and  Network  Solutions.  Upon  obtaining 
customer  approval,  it  is  now  the  responsibility  of  the  team  to  define  what  physical  assets 
are  necessary  to  execute  the  project.  As  with  the  search  through  existing  scenarios,  the 
team  should  look  at  the  asset  inventory  which  already  exists.  The  team  should  take 
advantage  of  this  and  identify  any  lacking  resource.  This  is  one  of  the  reasons  that  costs 
are  broken  into  two  sections.  It  is  not  until  the  requirements  and  scenario  have  been 
defined  that  the  Project  Lead  will  have  a  solid  idea  of  the  hardware,  software,  network, 
and  personnel,  resources  which  will  be  needed. 


Key  Players 

Inputs 

Outputs 

Developer 

Project  Lead 

Scenario 

Tangible  needs  list 

Table  35.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  Hardware,  Software,  and 

Network  Solutions  Step 


(2)  Identify  Project  Resources.  Given  the  development  of  project 
requirements,  scenario,  and  hardware  and  software  assets  needed,  the  Project  Lead  and 
Supervisor  have  a  relatively  clear  vision  of  the  level  of  effort  involved  in  the  analysis. 
Furthermore,  they  have  a  view  of  what  types  of  resources  are  needed,  such  as  software 
engineers,  analysts,  network  technicians,  and  statisticians.  The  supervisor  is  responsible 
for  assuring  availability  of  the  human  assets  required  for  the  project.  By  involving  the 
supervisor,  the  Project  Lead  gains  commitment  from  management  that  the  project  at  hand 
shall  be  supported. 


Key  Players 

Inputs 

Outputs 

Project  Lead 
Supervisor 

Scenario 

Hardware,  Software  and 
Network  Resources 

Staffing 

Management  Commitment 

Table  36.  Key  Players,  Inputs,  and  Outputs  for  the  Identify  Project  Resources  Sub-Step 
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b.  System  Generation 


The  system  generation  step  translates  the  storyboard  scenario  defined  in 
previous  steps  into  a  software  and  hardware  solution.  Application  of  the  functional 
decomposition  at  this  step  will  allow  the  developers  who  are  responsible  for  developing 
the  simulated  environment  to  establish  it  in  a  modular  fashion  in  order  to  meet  project 
demands. 
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Figure  29. 


System  Generation  Step  of  the  Execution  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 
Developers 

Resource  Identification 

Hardened  Resources 
Scenario  Load 

Table  37.  Key  Players,  Inputs,  and  Outputs  for  the  System  Generation  Step 


(1)  Acquire  Identified  Hardware,  Software,  and  Network.  Lead  time  is 
necessary  to  acquire  the  software  and  hardware  assets  identified.  It  is  optimal  to  use  this 
lead  time  in  parallel  with  previous  steps  but  is  not  always  possible  given  that  the 
simulation  environment,  which  includes  a  list  of  physical  assets  involved  in  conducting 
the  simulation  as  well  as  software  to  be  installed  to  run  the  simulation,  will  not  have  been 
identified  until  the  last  minute.  This  step  simply  consists  of  ordering  and  obtaining  all 
tangible  assets  for  the  project. 


Key  Players 

Inputs 

Outputs 

Project  Lead 
Financial  Manager 

Hardware,  Software, 
Network  Equipment  Needs 

Acquisition  of  Stated  Assets 

Table  38.  Key  Players,  Inputs,  and  Outputs  for  the  Acquire  Identified  Hardware,  Software 

and  Network  Step 
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(2)  Train  Staff.  The  Project  Lead  must  also  assure  that  the  staff  has 
received  relevant  training  on  the  assets  used  on  the  project.  By  not  doing  so,  the  setup  and 
execution  of  the  analyses  will  likely  be  error  prone  and  could  lead  to  negative  results. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Project  Analyst 

Team  Members 

All  hardware  and  software 

resources 

Knowledgeable  staff 

Table  39.  Key  Players,  Inputs,  and  Outputs  for  the  Train  Staff  Sub-Step 


(3)  Configure  Environment.  After  obtaining  the  assets  and  training  the 
staff,  the  environment  must  be  established.  This  includes  loading  software  on  machines, 
documenting  changes  to  be  recorded  in  the  configuration  management  system,  baselining 
all  machines  for  quick  recovery  should  it  be  needed,  and  developing  processes,  policies, 
and  procedures  in  order  to  conduct  the  analyses. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

System  Analyst 

Team  Members 

Hardware,  Software, 
Network,  and  Human 
Resources 

Operational  Environment 
which  is  configuration 
managed  and  has 
appropriate  documentation 

Table  40.  Key  Players,  Inputs,  and  Outputs  for  the  Configure  Environment  Sub-Step 


(4)  Develop  Procedures.  While  configuring  the  environment,  the  Systems 
Engineer  should  institute  a  systematic,  disciplined  approach,  documenting  every  step 
taken,  including  steps  which  generate  errors.  In  doing  so,  the  project  documentation  will 
include  a  troubleshooting  list.  The  procedures  developed  should  include  steps  for 
installing  and  configuring  needed  software  in  addition  to  steps  for  running  the  software. 
This  will  be  used  by  members  who  will  be  participating  in  the  scenario’s  execution. 
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Key  Players 

Inputs 

Outputs 

Team  Members 

Installations 

Configuration 

Documentation  of  change 
control  as  well  as  process 
steps  for  execution 

Table  41.  Key  Players,  Inputs,  and  Outputs  for  the  Develop  Procedures  Sub-Step 


c.  System  Execution 

The  System  Execution  process,  in  theory,  should  be  relatively 
straightforward  provided  ample  consideration  was  given  to  the  previous  steps.  Executing 
the  environment  consists  of  running  the  scenario  and  verifying  that  the  data  collected 
addresses  the  requirements  and  metrics. 
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Figure  30.  System  Execution  step  of  the  Execution  Phase 
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Table  42.  Key  Players,  Inputs,  and  Outputs  for  the  System  Execution  Step 


(1)  Run  Scenario(s).  By  executing  in  accordance  with  policies  and 
procedures  documented  in  the  previous  step  the  scenario  running  process  should  be 
straightforward.  Duration  depends  on  scenario  timeline,  number  of  humans  in  the  loop 
and  scenario  iterations. 
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Key  Players 

Inputs 

Outputs 

Players 

Project  Lead 

Project  Analyst 

Physical  System 
Personnel  Resources 

Scenario  Execution 

Table  43.  Key  Players,  Inputs,  and  Outputs  for  the  Run  Scenario(s)  Sub-Step 


(2)  Verify  Data  Collection.  It  is  recommended  to  verify  data  collection 
after  an  initial  scenario  run.  This  includes  checking  for  the  output  data  fields  and 
verifying  their  application  to  the  requirements  and  metrics,  as  well  as  considering  the 
values  being  returned  and  whether  or  not  they  seem  realistic.  If  the  wrong  values  are 
being  collected,  the  data  collection  plan  should  be  cross  referenced  for  completeness  and 
the  data  collection  tool  configuration  should  be  verified.  Any  unrealistic  return  values 
should  be  analyzed,  leading  to  a  review  of  input  values,  the  impacts  of  any  randomizer 
involved,  and  consideration  to  the  standard  deviation  of  outer  bounds. 


Key  Players 

Inputs 

Outputs 

Project  Lead 

Project  Analyst 

Scenario  Data 

Information  applicable  to 
analyses 

Table  44.  Key  Players,  Inputs,  and  Outputs  for  the  Verify  Data  Collection  Sub-Step 


When  perfonning  a  data  collection  validation  check,  consider  that  the 
initial  settings,  such  as  the  initial  seed  produced  by  a  random  number  generator  used  may 
create  extreme,  yet  valid  results.  Further  iterations  of  the  scenario  are  necessary  until  the 
analyst  is  satisfied  that  the  results  yielded  are  appropriate. 

d.  Document  Risks,  Issues,  &  Constraints 


Figure  3 1 .  Document  Risks,  Issues  &  Constraints  step  of  the  Execution  Phase 
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Key  Players 

Inputs 

Outputs 

All 

All  Phase  Activities 

Updated  open  statements  of 
concern  for  stakeholders  to 
consider 

Table  45.  Key  Players,  Inputs,  and  Outputs  for  the  Document  Risks,  Issues  &  Constraints 
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Figure  32.  Configuration  Management  Step  of  the  Execution  Phase 
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Project  Lead 
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Existing  artifacts  to  date 

Controlled  method  to 
preserve  data 

Table  46.  Key  Players,  Inputs,  and  Outputs  for  the  Configuration  Management  Step 


f.  Perform  Customer  Demonstration 

The  customer  demonstration  provides  to  the  customer  an  overview  of  the 
scenario  being  executed  and  details  of  the  data  being  collected.  The  customer  should  not 
expect  any  analyses  to  be  perfonned  at  this  point  in  time.  The  goal  of  this  step  is  to  assure 
that  all  considerations  have  been  made  regarding  customer  requirements,  that  the  data 
being  collected  will  be  useful  for  analysis  and  that  no  scenario  reworks,  or  data  collection 
adjustments  are  necessary. 
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Figure  33.  Perfomi  Customer  Demonstration  Step  of  the  Execution  Phase 
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Key  Players 

Inputs 

Outputs 

Project  Lead 

Project  Analyst 

Execution  of  a  single 
scenario 

Customer  feedback  before 
performing  multiple 
iterations  of  the  scenario 

Table  47.  Key  Players,  Inputs,  and  Outputs  for  the  Perform  Customer  Demonstration  Step 


4.  Analysis  Phase 

Analysis  begins  at  the  point  of  scenario  creation  and  design.  By  doing  so,  the 
analyst  has  a  good  idea  of  what  the  initial  scenario  state  is  and  what  infonnation  can 
ultimately  be  derived  from  it.  The  main  brunt  of  the  task  occurs  during  and  after  scenario 
runs.  It  is  the  analyst’s  job  to  receive  and  process  information,  turn  it  into  useful  data, 
perform  analysis  on  the  data,  then  format  and  report  the  results. 


Figure  34.  Analysis  Phase 
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Table  48.  Key  Players,  Inputs,  and  Outputs  for  the  Analysis  Phase 


a.  Process  Information 

There  is  a  major  distinction  between  infonnation  and  data,  as  all  things 
that  are  received  are  considered  data  while  only  the  relevant  items  are  further  refined  to 
become  information.  The  amount  of  information  available  becomes  significant  as  better 
data  collection  plans  are  developed  and  implemented.  However,  it  is  important  to  note 
that  there  may  not  be  enough  information  based  on  the  data  provided,  or  when 
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information  appears  unreliable,  the  data  collection  must  be  expanded  in  order  to 
investigate  any  dependencies  which  may  be  skewing  the  results. 
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Figure  35.  Process  Information  Step  of  the  Analysis  Phase 
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Table  49.  Key  Players,  Inputs,  and  Outputs  for  the  Process  Information  Step 


b.  Analyze  Information 

Once  the  correct  amount  and  fidelity  of  information  has  been  collected, 
the  analyst  performs  the  appropriate  analyses.  Methods  of  analysis  are  beyond  the  scope 
of  this  document  and  therefore  shall  not  be  discussed. 
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Figure  36.  Analyze  Information  Step  of  the  Analysis  Phase 
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Table  50.  Key  Players,  Inputs,  and  Outputs  for  the  Analyze  Infonnation  step 


c.  Document  Risks,  Issues,  Constraints 
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Figure  37. 


Document  Risks,  Issues  &  Constraints  Step  for  the  Analysis  Phase 
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Table  5 1 .  Key  Players,  Inputs,  and  Outputs  for  the  Document  Risks,  Issues  &  Constraints 

Step 
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Figure  38.  Configuration  Management  step  of  the  Analysis  Phase 


Key  Players 

Inputs 

Outputs 

Project  Lead 
Developers 

All  existing  artifacts 

Controlled  method  to 
preserve  data 

Table  52.  Key  Players,  Inputs,  and  Outputs  for  the  Configuration  Management  Step 


d.  Customer  Presentation 

When  appropriate,  the  analyst  shall  create  visual  representations  as  well  as 
written  reports,  spreadsheets,  and  other  pertinent  documents  on  the  results.  It  is 
recommended  that  the  Project  Lead  and  Systems  Engineer  be  present  during  reporting  as 
stakeholders  may  have  questions  regarding  the  means  by  which  results  were  derived. 
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Figure  39.  Customer  Presentation  Step  of  the  Analysis  Phase 
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Table  53.  Key  Players,  Inputs,  and  Outputs  for  the  Customer  Presentation  Step 
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III.  SUMMARY 


In  summary,  Figure  40  presents  the  current  Architectural  process  proposed  to 
address  the  need  for  a  standardized  process  for  performing  analysis  within  the 
Operational  Simulation  &  Analysis  branch.  Sources  of  information  referenced  in  the 
development  of  this  process  range  from  publications  related  to  the  discipline  of  System’s 
Engineering,  Software  Engineering,  and  Operations  Research,  in  addition  to  personal 
experience  from  over  a  decade  in  this  specific  field  which  has  served  to  provide  the 
majority  of  influence  and  contribution  to  its  creation. 


Figure  40.  Architectural  Process 


Having  such  a  plan  is  important  to  the  OS&A  branch  and  ARDEC  as  a  whole 
because  it  documents  decisions,  facilitates  communication  among  stakeholders,  and 
maintains  a  record  of  scope,  cost,  and  schedule  baselines.  By  instituting  a  standardized 
process,  the  OS&A  branch  would  ensure  that  results  based  on  the  Analysis  Project  Plan 


51 


are  reusable,  allow  for  configuration  management,  better  management  of  overall 
resources,  and  better  validation  and  verification. 

The  process  is  presented  in  four  phases,  Initiation,  Planning,  Execution,  and 
Analysis.  Within  each  phase  are  sequential  steps.  The  project  lead  has  the  flexibility  to 
add  or  omit  steps  as  required  by  the  particular  project  and  the  OS&A  branch  as  a  whole  is 
requested  to  contribute  suggestions  to  this  process  over  time  in  order  to  provide  the  best 
process  possible  for  the  purposes  of  performing  analysis  through  simulation. 

The  key  aspects  in  the  proposed  process  consist  of  the  following: 

Process  Flexibility:  Adapting  to  changes  as  they  become  prevalent  in  addition  to 
incorporating  emerging  best  business  practices  ensures  that  the  process  will  continue  to 
benefit  the  project  lead  and  that  the  project  lead,  in  turn,  will  embrace  the  process. 

Emphasis  on  Deriving  Level  of  Effort  /  Aggregation  in  the  Initiation  Phase:  A 
shortcoming  observed  through  personal  observation  of  business  processes  indicates  that 
the  current  business  model  frequently  underestimates  scope,  time,  and  cost  considerations 
which  ultimately  lead  to  overruns.  In  establishing  the  level  of  effort  required  for  project 
success  through  determining  aggregation  level  of  the  analyses  prior  to  executing  a 
systems  approach,  the  project  lead  gains  the  ability  to  verify  that  the  customer’s  need  is  in 
alignment  with  the  amount  of  funding  and  time  the  customer  is  willing  to  commit. 

The  Planning  Phase  marks  the  kickoff  of  the  Systems  Approach:  During  the 
Planning  Phase  the  discipline  of  the  systems  approach  is  executed.  The  outputs  of  the 
Initiation  Phase  provide  the  basis  for  the  Planning  phase  such  that  the  project  lead  has 
identified  the  customers  and  stakeholders  from  which  to  conduct  requirements  elicitation, 
then  perform  prioritization,  decomposition,  and  metric  development  activities.  At  the 
completion  of  these  activities  reliable  estimates  of  time  and  cost  are  understood, 
presented,  and  resolved  through  the  customer. 

Establish  Retainer  Funding  to  Detennine  Project  Feasibility:  Through  establishing 
a  dual  payment  system,  or  retainer  funding  scheme,  both  the  customer  and  the 
Operational  Simulation  &  Analysis  branch  are  ensured  the  highest  degree  of  success  with 
minimal  waste  in  terms  of  cost  and  time.  The  intent  of  the  retainer  funding  is  to  allow  the 
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project  lead  to  conduct  the  Project  Initiation  and  Planning  phases  and  detennine  if 
proceeding  to  the  Execution  and  Analysis  phases  as  prescribed  are  feasible.  As  an 
illustration,  consider  the  activities  conducted  when  an  individual  purchases  a  home.  In  the 
majority  of  cases,  prospective  buyers  prefer  to  make  the  investment  in  a  home  inspection 
in  order  to  decide  whether  to  proceed  with  the  purchase,  renegotiate  terms,  or  tenninate 
the  contract.  The  prospective  buyer  understands  that  this  is  a  non-refundable  investment 
but  finds  the  cost  of  the  investment  preferable  to  the  potential  consequence  of  not  having 
the  inspection  done  at  all. 

Iterative  Risk  Management:  A  continual  feedback  loop  between  the  project  lead, 
customer,  and  stakeholder  ensure  the  customer’s  expectations  are  being  met  while 
unexpected  consequences  that  arise  throughout  the  lifetime  of  the  project  are 
communicated  between  all  parties  involved.  Risk  management  assures  that  all  parties 
have  collectively  participated  in  the  resolution  of  unforeseen  circumstances  which  may 
otherwise  halt  project  execution  and  impair  overall  project  success. 

Iterative  Configuration  Management:  Adopting  and  adhering  to  a  configuration 
management  scheme  provide  a  benefit  to  the  entire  OS&A  branch  and,  in  turn,  future 
customers  through  the  practice  of  reuse.  The  ability  to  consult  an  archive  of  previous 
exercises  for  reuse  in  a  current  project  will  aid  in  minimizing  cost  and  schedule 
requirements. 
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IV.  CONCLUSION 


In  conclusion,  past  experience  has  shown  that  adopting  any  structure  as  opposed 
to  operating  in  an  ad  hoc  fashion  is  beneficial  to  branch  success.  This  thesis  has  presented 
a  process  with  specific  application  to  the  field  of  analyses  through  modeling  and 
simulation.  Furthennore,  this  thesis  has  also  defined  the  roles  and  described  activities  key 
players  enact  throughout  its  progression.  By  establishing  a  flexible  process  where 
communication  of  the  problem  is  precise,  the  magnitude  of  the  solution  is  relevant  and 
reliable,  and  the  tools  and  personnel  to  execute  the  analysis  are  employed  at  the  right 
times,  the  Operational  Simulation  and  Analysis  branch  will  continue  to  establish  their 
role  in  ARDEC’s  overall  objective  striving  to  support  the  RDECOM  mission  of  getting 
the  right  technology  to  the  right  place,  at  the  right  time  for  the  War  fighter  (U.S.  Army 
ARDEC  2001). 
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